fix: signed-shift overflow in hdr_calculate_bucket_config (UBSan, found by fuzzing) - #145

🤖 Automated first-pass review — a human maintainer's review is still required before merge.

The core fix looks right. Moving the > 61 guard above the sub_bucket_mask shift removes the UB without changing anything for valid configs, and lowest_discernible_value > highest_trackable_value / 2 is equivalent to the old lowest * 2 > highest for every input that reaches it (lowest >= 1 is already checked above it, and C's truncation toward zero makes the two agree for negative highest_trackable_value as well). Worth noting the 61 threshold is load-bearing in a second place: buckets_needed_to_cover_value() computes ((int64_t) sub_bucket_count) << unit_magnitude, whose top bit is unit_magnitude + sub_bucket_half_count_magnitude + 1, so 61 is exactly what keeps that at bit 62 and representable — it shouldn't be relaxed. No ABI surface is touched here (no struct hdr_histogram layout change, no public signature change), so no SOVERSION step is implicated.

The three hdr_free(h) → hdr_close(h) changes in the aggregate branch of hdr_decode_compressed_v0/v1/v2 are real leaks of the counts array, and they're the same bug #122 fixed on the error paths of those same three functions while missing the success path — the cleanup block already read hdr_close(h) two branches up. That's a user-visible fix in the log-decode path, and the title and description mention only the shift overflow; it should be called out explicitly, since it's the part a downstream consumer would care about in a release note.

Two smaller things. memset(cfg, 0, sizeof(*cfg)) establishes a new guarantee about a public out-param on the EINVAL path, and test_bucket_config_reject_defines_cfg now pins that guarantee, but nothing in hdr_histogram.h states it — either document it next to hdr_calculate_bucket_config or drop it, since no in-tree caller reads cfg after a non-zero return and hdr_init returns immediately. Separately, the new free(buffer) / free(data) calls in the log tests are freeing memory that hdr_encode_compressed and hdr_log_encode allocated via hdr_malloc/hdr_calloc; that's consistent with what those tests already did and harmless with the default allocator, but it's a mismatched pair for anyone building with a custom HDR_MALLOC_INCLUDE.

The sanitizers job is the most useful part of this PR beyond the fix itself: the per-PR fuzzing gate runs ASan only and UBSan lands only in the weekly batch, which is precisely why this class of bug surfaced late. Be clear about what it doesn't cover, though — it's Linux/x64/gcc only, so it does nothing for the BSD/musl/32-bit build breakages that are historically this project's more frequent failure mode. The .hlog reproducer needs no build-script change since .clusterfuzzlite/build.sh already globs test/*.hlog into the seed corpus, but the deterministic per-PR coverage here really comes from test_bucket_config_shift_overflow, not from the corpus file.

I read the sources to check the arithmetic equivalence and the recorder ownership — hdr_interval_recorder_destroy does close both active and inactive, and the leftover sample1 in reset_histogram_on_sample_and_recycle genuinely isn't held by the recorder, so the added hdr_close calls don't introduce a double free. I have not built or run the suite under ASan+UBSan myself, so the "clean under both sanitizers" claim rests on the author's own verification and on the new job.